BIALEK Bill  Princeton  PHASE DIAGRAMS and Real Neural Nets 27 Feb 2013

(this is the second talk in the MBC symposium today on Dynamics in Neural Nets)

Surya Ganguli introduces Bill;
Surya uses his biophys book for his biophy course at Stanford.

are emergent phenom  real  (something that emerges from collective  behavior)

is it kind of like birds flocking (but metaphorical or is it real?)

he shows  8 of his papers  on related topics... (but does not discuss those)

today will talk about phase diagrams...

slide 1 is a phase diagram  (think of state phases in water)
(but water is horribly complicated)

Y axis is pressure vs temperature on x axis;
at different regions you have either  solid, liquid  gas with

sharp boundaries btwn solid   and liquid and gas/ vapor
this works if you  have  a macroscopic collection of stuff... (not just a few molecules)

these are bulk properties of big collections of stuff..

what phase you are in... tells u a lot about the behavior of the substance, eg

liquid...navier stokes eqn applies to liquids:   viscosity and density  (only applies to fluids)

even though difft substances are difft:

there are universal  rules  that  govern  transitions  near critical points
that govern the properties of large groups...

how abt neural nets...?  Say..

X axis is the strength of excitatory connections and
Y axis = inhibitory strength

somewhere in btwn (in the middle of the scatterplot)  there should be a balance...  BUT
if strength of excitatory connex is too great then perhaps get epilepsy  and 
if inhibitory connex are too strong  then get  boredom (he says)  (or coma, I say).

now what really belongs on these axes...?

connection strengths?
all indiv connex ?

what characterizes the phases of such a network?
faithful recall of memories?
rhythms?
reliable responses to sensory inputs?

where ARE the real networks? possibilities:
each circuit is deep in one phase  OR
circuits move across phase boundaries OR
circuits sit near phase boundaries...

*****************

three ideas  that we think about... (in terms of methods to approach this)

a)  just count (firing neurons)

b) make a detailed model (of network)

c)  imagine an ensemble of networks..

a) above works surprisingly well... this will be the emphasis of his talk today...

*****
JUST COUNT NEURONS  (each neuron is binary:  in any given
time bin (eg  20 ms):  it either fired at least once or it did not.

he gives a brief review of statist mechanics.

experiment one: (by his expt colleagues)

record from   100+  neurons in salamander retina...

electrode arrays...   record from all the ret ganglion cells
record all the spikes...

perhaps recod from about 150 in an area where there are abt 250 neurons..

as the retina responds to naturalistic movies:   (similar to Steve Baccus work)

bin the time data into 20 ms bins...  either responds or not  (spikes or not)

so  get an N bit word   try N = 20  over time

every moment in time (for each bin)...  have a 20 bit word...

he plots the probability of a given word on Y axis  (zipf plot)

vs the rank of word is on X axis...

low rank (v few neurons firing: most are not firing)
is far more common than high rank  at lower right...

how many states are there that have a partic energy?

states that have high energy  have lower probability
(corresp to higher temperature)

give system more energy... then there are more available states...

tradeoff  btwn  states of high energy  but lesser probability.

*****

boltzmann distrib...

1)  measure the probab of each word  (just by counting)
2) convert probabilities to energies

the decr probab of indiv states is almost perfected matched to the incr number of states with lower probab.

means network is sitting near a critical point..

*****
many steps to make this work...
model may predict   eg  hopfield style patterns

we have strong hints that this real nework is sitting near a critical point (or critical surface).

critical points maximize the dynamic range of surprise...

take - log(prob) of what  you saw... that's how surprised you are...

now u can do an expt... u can actually count  how often real networks
 take on properites and measure probs of states  (eg with salamander retina)...

*************************

Jay mcC   how does this apply to the movies themselves...
(ie their intrinsic statistics.)

A: cortex is certainly influenced by  distrib of its inputs,
distrib of its own states (and modulation/ adaptation)
and distrib of connections...

easier to talk abt inputs to retina but cortex has inputs too.